ceph-crush algorithm(二)

2017-06-06 16:23:40
源码文件

crush目录下的源码文件如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
CrushCompiler.cc	
CrushCompiler.h	
CrushLocation.cc	
CrushLocation.h	
CrushTester.cc	
CrushTester.h	
CrushTreeDumper.h	
CrushWrapper.cc	
CrushWrapper.h	
CrushWrapper.i	
builder.c	
builder.h	
crush.c	
crush.h	
crush_compat.h	
crush_ln_table.h	
grammar.h	
hash.c	
hash.h	
mapper.c	
mapper.h	
old_sample.txt	
sample.txt	
types.h

CRUSH maps

crush maps组成: osd list,Bucket list ,rule list

Device

1
2
3
4
5
6
7
8
device 0 osd.0
device 1 osd.1
device 2 osd.2
device 3 osd.3
device 4 osd.4
device 5 osd.5
device 6 osd.6
device 7 osd.7

Bucket

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
host ceph-osd-ssd-server-1 {
           id -1
           alg straw
           hash 0
           item osd.0 weight 1.00
           item osd.1 weight 1.00
   }
   host ceph-osd-ssd-server-2 {
           id -2
           alg straw
           hash 0
           item osd.2 weight 1.00
           item osd.3 weight 1.00
   }
  host ceph-osd-platter-server-1 {
           id -3
           alg straw
           hash 0
           item osd.4 weight 1.00
           item osd.5 weight 1.00
   }
   host ceph-osd-platter-server-2 {
           id -4
           alg straw
           hash 0
           item osd.6 weight 1.00
           item osd.7 weight 1.00
   }
   root platter {
           id -5
           alg straw
           hash 0
           item ceph-osd-platter-server-1 weight 2.00
           item ceph-osd-platter-server-2 weight 2.00
   }
   root ssd {
           id -6
           alg straw
           hash 0
           item ceph-osd-ssd-server-1 weight 2.00
           item ceph-osd-ssd-server-2 weight 2.00
   }

Rule

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
rule data {
              ruleset 0
              type replicated
              min_size 2
              max_size 2
              step take platter
              step chooseleaf firstn 0 type host
              step emit
      }
      rule metadata {
              ruleset 1
              type replicated
              min_size 0
              max_size 10
              step take platter
              step chooseleaf firstn 0 type host
              step emit
      }
      rule rbd {
              ruleset 2
              type replicated
              min_size 0
              max_size 10
              step take platter
              step chooseleaf firstn 0 type host
              step emit
      }
      rule platter {
              ruleset 3
              type replicated
              min_size 0
              max_size 10
              step take platter
              step chooseleaf firstn 0 type host
              step emit
      }
      rule ssd {
              ruleset 4
              type replicated
              min_size 0
              max_size 4
              step take ssd
              step chooseleaf firstn 0 type host
              step emit
      }
      rule ssd-primary {
              ruleset 5
              type replicated
              min_size 5
              max_size 10
              step take ssd
              step chooseleaf firstn 1 type host
              step emit
              step take platter
              step chooseleaf firstn -1 type host
              step emit
      }

Do Rule
1
2
3
4
5
6
7
8
9
void do_rule(int rule, int x, vector<int>& out, int maxout,
	       const WeightVector& weight,
	       uint64_t choose_args_index) const {
    int rawout[maxout];
    char work[crush_work_size(crush, maxout)];
    crush_init_workspace(crush, work);
    crush_choose_arg_map arg_map = choose_args_get(choose_args_index);
    int numrep = crush_do_rule(crush, rule, x, rawout, maxout, &weight[0],
			       weight.size(), work, arg_map.args);

@rule: 使用的crush_rule在crush_map的rules列表中所在index
@x: 输入Hash ID，object_id或者pg_id)
@out: 输出Device ID列表
@maxout: 在输出Device ID的个数，副本的个数
@weight: 输出Device列表对应的权重

具体工作调用crush_do_rule完成

Mapper.c

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
/**
 * crush_do_rule - calculate a mapping with the given input and rule
 * @map: the crush_map
 * @ruleno: the rule id
 * @x: hash input
 * @result: pointer to result vector
 * @result_max: maximum result size
 * @weight: weight vector (for map leaves)
 * @weight_max: size of weight vector
 * @cwin: Pointer to at least map->working_size bytes of memory or NULL.
 */
int crush_do_rule(const struct crush_map *map,
		  int ruleno, int x, int *result, int result_max,
		  const __u32 *weight, int weight_max,
		  void *cwin, const struct crush_choose_arg *choose_args)

值得说的变量emit 通常用在规则的结束,同时可以被用在在形相同规则下选择不同的树.更多详细信息看这里

ref
CEPH CRUSH algorithm source code analysis
ceph的CRUSH算法的源码分析
大话Ceph–CRUSH那点事儿
Ceph源码目录架构

您的鼓励是我写作最大的动力

俗话说，投资效率是最好的投资。 如果您感觉我的文章质量不错，读后收获很大，预计能为您提高 10% 的工作效率，不妨小额捐助我一下，让我有动力继续写出更多好文章。